Skip to content

Conversation

@weanti
Copy link

@weanti weanti commented Jun 26, 2025

Fix constructing frame offsets for encapsulated pixel data.
Fix reading frames from encapsulated pixel data.

Remove restrictions for bits stored value, because it is not valid in all cases e.g. CT.

Antal Ispanovity added 3 commits June 26, 2025 15:03
… specific e.g. in case of CT it can be 12,13,14,15,16
…ally not read, undefined length was assumed

use more intuitive offset calculation: previous offset + tag and length size + current length
@jcupitt
Copy link
Collaborator

jcupitt commented Jun 30, 2025

Hi @weanti,

This looks great! Thank you for doing this work.

Could you explain what kinds of DICOM you are working with? Do you have some sample DICOMs I could use for testing?

You probably saw the other PR (#102): we should probably merge that first, since it covers some of the same ground.

@weanti
Copy link
Author

weanti commented Jun 30, 2025

Hi @weanti,

This looks great! Thank you for doing this work.

Could you explain what kinds of DICOM you are working with? Do you have some sample DICOMs I could use for testing?

You probably saw the other PR (#102): we should probably merge that first, since it covers some of the same ground.

Hi,

thank you for your response.
In the meantime I did further testing and I found problems. Memory overwrites and offset handling problems.
Sure, I try to attach or send some test files. Btw I used JPEG2000 encoded DICOM images.
First I need to polish this PR.

@jcupitt
Copy link
Collaborator

jcupitt commented Jun 30, 2025

OK, let's tag this as a draft for now.

@jcupitt jcupitt marked this pull request as draft June 30, 2025 15:01
fix indexing problem when creating offsets table
fix frame size calculation: bits allocated was not considered
@weanti
Copy link
Author

weanti commented Jul 1, 2025

I think I solved the problems.
I have attached a bunch of DICOM files. See the readme.txt for important properties.
UPDATE: uoloading files doesn't seem to work. Here is a link for a shared zip file: https://drive.google.com/file/d/1UPgq89YLx94XyiuGOupR2t0LtgQ4VnkS/view?usp=sharing

@weanti weanti marked this pull request as ready for review July 3, 2025 04:18
@weanti
Copy link
Author

weanti commented Aug 27, 2025

@jcupitt What's your opinion about this PR? The other PR (the one before this) is updated. If that is merged, then I'll rebase and update my PR.

@jcupitt
Copy link
Collaborator

jcupitt commented Aug 27, 2025

Hi @weanti, sorry, I was on holiday and then got distracted by other projects.

I'll look this over again now.

@jcupitt
Copy link
Collaborator

jcupitt commented Aug 27, 2025

... I saw one final tiny issue in #102, when that's resolved I'll look at this more closely.

@weanti
Copy link
Author

weanti commented Aug 27, 2025

... I saw one final tiny issue in #102, when that's resolved I'll look at this more closely.

Thank you for the feedback.

@weanti
Copy link
Author

weanti commented Sep 30, 2025

Resolved conflicts.

@jcupitt
Copy link
Collaborator

jcupitt commented Sep 30, 2025

I'll read this tomorrow. Thanks for the update!

@jcupitt
Copy link
Collaborator

jcupitt commented Oct 1, 2025

I downloaded the zip file, these are useful samples!

What's the licence? Could we add them to the test suite?

@weanti
Copy link
Author

weanti commented Oct 1, 2025

I downloaded the zip file, these are useful samples!

What's the licence? Could we add them to the test suite?
I downloaded the compsamples_j2k archive (a set of jpeg 2000 compressed images) from ftp://medical.nema.org/MEDICAL/Dicom/DataSets/WG04
This includes the SC1_J2KR and VL1_J2KR images.
Another set of images was dwnloaded from https://www.dcmtk.org/download/images/
nema97cd.zip.
This contains im309 in gems/dlx folder.
I think these are public domain.
I try to find the source of the segmentation_j2k.

@jcupitt
Copy link
Collaborator

jcupitt commented Oct 1, 2025

Great! Still to do:

  • add some tests
  • add a line to the changelog and credit yourself
  • reformat for libdicom style
  • some minor restructuring, as noted

@fedorov
Copy link
Member

fedorov commented Oct 2, 2025

What's the licence? Could we add them to the test suite?

I downloaded the compsamples_j2k archive (a set of jpeg 2000 compressed images) from ftp://medical.nema.org/MEDICAL/Dicom/DataSets/WG04
This includes the SC1_J2KR and VL1_J2KR images.
Another set of images was dwnloaded from https://www.dcmtk.org/download/images/
nema97cd.zip.
This contains im309 in gems/dlx folder.
I think these are public domain.
I try to find the source of the segmentation_j2k.

@dclunie can you confirm what is the license for the images available from the NEMA FTP server?

@michaelonken @jriesmeier how about the images shared in https://www.dcmtk.org/download/images/ ?

@dclunie
Copy link

dclunie commented Oct 2, 2025

There is no specified license for the CAR97, NEMA97, or WG04 images - we had intended all of these to be publicly usable without restriction (we gave away the CDs at meetings, and shared the images online by anonymous ftp), but never defined a license.

@michaelonken
Copy link

michaelonken commented Oct 3, 2025

David said it all ☝️

[Edit: The folder ddsm/ has its own README. The collection of images stems from the University of South Florida and has originally been provided in non-DICOM format. We converted it (I don't remember the exact context) and offered to host these images on the OFFIS servers (which they were fine with). I found a copy of the now-offline original website in the Internet archive. The images have been part of a research grant; maybe you find more information if you dig through the site.]

@weanti
Copy link
Author

weanti commented Oct 6, 2025

Great! Still to do:

  • add some tests
  • add a line to the changelog and credit yourself
  • reformat for libdicom style
  • some minor restructuring, as noted

I'm working on tests. Will take some time due to limited availability.

@weanti
Copy link
Author

weanti commented Oct 9, 2025

Added some tests. These are rather functional test, because the implementation that handles encapsulated pixel data is not on the public API.
The test data is generated and may not be DICOM conformant e.g. not really loadable by real applications. The tests focus on the "happy path". Shall I add some tests for the error cases as well?

@weanti
Copy link
Author

weanti commented Nov 7, 2025

@jcupitt Shall we proceed with this PR?

@jcupitt
Copy link
Collaborator

jcupitt commented Nov 7, 2025

Sorry @weanti I got distracted again.

I'll do a final review now.

Copy link
Collaborator

@jcupitt jcupitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry :( one more thing.

if (!read_tag(&state, &tag, &position) ||
!read_uint32(&state, &length, &position)) {
return false;
// each frame may consist of several fragments, so we need to scan each fragment to find the next frame
Copy link
Collaborator

@jcupitt jcupitt Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need this complicated loop. I think (I think! I hope!) all you need to do is scan num_frames items and collect the offsets. This will collect the right number of frames whether or not frames are split into many items.

How about changing this back to:

turn false;
        }
    } else {
        // the BOT is missing, we must scan pixeldata to find the position of
        // each frame
   
        dcm_log_info("building Offset Table from Pixel Data");
   
        // 0 in the BOT is the offset to the start of frame 1, ie. here
        *first_frame_offset = position;

        position = 0;
        for (int i = 0; i < num_frames; i++) {
            if (!read_tag(&state, &tag, &position) ||
                !read_uint32(&state, &length, &position)) {
                return false;
            }

            if (tag == TAG_SQ_DELIM) {
                dcm_error_set(error, DCM_ERROR_CODE_PARSE,
                              "reading BasicOffsetTable failed",
                              "too few frames in PixelData");
                return false;
            }
         
            if (tag != TAG_ITEM) {
                dcm_error_set(error, DCM_ERROR_CODE_PARSE,
                              "building BasicOffsetTable failed",
                              "frame Item #%d has wrong tag '%08x'",
                              i + 1,
                              tag);
                return false;
            }
            
            // step back to the start of the item for this frame
            offsets[i] = position - 8;
            
            // and seek forward over the value
            if (!dcm_seekcur(&state, length, &position)) {
                return false;
            }
        }
    }

    return true;
}

ie. just removing the check for the end of sequence tag.

I suppose we could check for end-of-sequence in the num_frames > 1 case, but I don't know if it'd add much.

fragment_length = 0;
char* fragment = value;
position = 0;
while(position < frame_end_offset) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@jcupitt
Copy link
Collaborator

jcupitt commented Nov 8, 2025

I'm still worried by the BOT builder, I think you don't need the new loop you made.

How about just reading num_frames tags, exactly as before. If num_frames is 1, then you'll just have the distance to the first frame. If it's more than one, then your assumption of one frame == one item will work fine. All you need to do is skip the "nest tag is delim" check we had at the end for the num_frames == 1 case.

One optimisation we could do would be to change this around to scan all items rather than scan all frames. We could record total frame length somewhere, then dcm_parse_encapsulated_frame() wouldn't need to make two passes over the data, it could just use the frame length we found in the BOT build. That's probably for a later PR though.

The new code isn't following the libdicom layout style exactly, but I can fix that after merge if you like (or do change it yourself, of course).

Otherwise this looks good!

return NULL;
}
dcm_seekcur(&state, fragment_length, &position);
*length += fragment_length;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose there's a potential uint32 overflow here. length should ideally be a uint64, though I don't suppose it matters much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants